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Subject: Sense and Avoid ELOS Definition 


Statement of Question/Issue: 

All manned aircraft are required to maintain vigilance to “see and avoid” other aircraft when 
operating in the National Air Space (NAS). Since unmanned aircraft do not have a pilot on-board 
the aircraft, they cannot literally comply with the “see and avoid” requirement beyond a short 
distance from the location of the unmanned pilot. Currently, unmanned aircraft are only allowed 
to operate within the NAS through means of segregation from manned aircraft. One primary 
reason is due to the lack of an approved/certified collision avoidance system for use on 
unmanned aircraft. No performance standards are presently defined for unmanned Sense and 
Avoid systems, and the FAA has no published approval criteria for a collision avoidance system. 

Discussion: 

Before the FAA can develop the necessary guidance (rules / regulations / policy) regarding the 
see-and-avoid requirements for Remotely Operated Aircraft (ROA), a concise understanding of 
the term “equivalent level of safety” must be attained. Since this term is open to interpretation, 
the ROA industry and FAA need to come to an agreement on how this term can be defined and 
applied for a safe and acceptable collision avoidance capability for unmanned aircraft. 

According to 14CFR 91.113, regardless of whether an aircraft is operating under visual flight 
rules (YFR) or instrument flight rules (IFR) the pilot in command is to remain vigilant to see and 
avoid other aircraft. Although this responsibility ultimately rests with the pilot, see and avoid is 
only one element within the overall traffic avoidance task. Traffic avoidance is accomplished 
through an integrated, overlapping conflict management process intended to minimize the risk of 
mid-air collisions. There are two major parts, conflict and collision avoidance. A conflict is any 
situation involving an aircraft and hazard in which the applicable separation minima may be 
compromised. A collision is a situation involving the physical contact between an aircraft and an 
airborne hazard. Airspace procedures and Air Traffic Management are used to create and 
maintain separation between aircraft to avoid conflict. Collision avoidance is the last layer of 
conflict management, is not considered part of separation provisions, and must activate when 
separation has been compromised. 
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See and avoid addresses collision avoidance and is the last line of defense used by pilots to avoid 
collisions with other aircraft and obstacles located in the air and on the surface. This distinction 
between conflict and collision avoidance, and the focus on collision avoidance will be important 
as the definition for sense and avoid safety equivalence is developed. 

Access 5 Project Position: 

The see and avoid requirement for unmanned aircraft must be understood prior to development 
of systems to provide that capability, and creation of certification criteria/policy to evaluate the 
adequacy of that capability. Defining an equivalent level of safety (ELOS) for sense and avoid is 
one of the first steps in understanding the requirement and developing a collision avoidance 
capability. 

Both manned and unmanned aircraft must meet the see and avoid requirement as stated in 14 
CFR 91.113. Manned aircraft use human vision, sometimes with cueing assistance, to see and 
avoid. However, since unmanned aircraft do not have a human on board, they need to have a 
sense and avoid capability that provides an ELOS to manned aircraft. The question then becomes 
- what is sense and avoid ELOS to manned aircraft? 

The attached document, Sense-and-Avoid Equivalent Level of Safety Definition for Unmanned 
Aircraft Systems, provides a functional level definition of see-and- avoid as it applies to 
unmanned aircraft. The sense and avoid ELOS definition is intended as a bridge between the see 
and avoid requirement and the system level requirements for unmanned aircraft sense and avoid 
systems. The discussions assume that this collision avoidance capability must be provided for 
both cooperative and non-cooperative traffic. Sense and avoid ELOS is defined in a rather 
abstract way, meaning that it is not technology or system specific, and the definition provides 
key parameters (and a context for those parameters) to focus the development of cooperative and 
non-cooperative sense and avoid system requirements. The ELOS definition, and to a lesser 
extent the sense and avoid system level requirements, will also provide a recommended set of 
functional standards to be submitted to the RTCA committee for unmanned aircraft sense and 
avoid systems. In addition, the system level requirements will be provided to the RTCA 
committee as a source of information for the development of sense and avoid system equipment 
requirements. The end goal is sufficient information for both the FAA and industry to develop 
sense and avoid systems for unmanned aircraft and get a favorable sense-and-avoid ELOS 
determination by the FAA, thus enabling certification and routine operations of unmanned 
aircraft in the NAS. 

A key point to note is that the complete answer to sense-and-avoid ELOS is a system or systems 
that address both cooperative and non-cooperative traffic situations. However, under certain 
conditions (e.g. in Class A airspace or above, with a certain type of control system, and perhaps 
some other limitations), it may be possible to obtain a favorable sense-and-avoid ELOS 
determination using a cooperative only system. 

The proposed definition and supporting analysis contained in Sense-and-Avoid Equivalent Level 
of Safety Definition for Unmanned Aircraft Systems, provides rationale for establishing objective 
performance criteria and standards for a sense-and-avoid capability necessary for routine 
operations in the NAS. 
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EXECUTIVE SUMMARY 


The ability of a pilot to detect other aircraft and avoid collisions has been the foundation of safe 
aircraft operations since the beginning of aviation. This process, most commonly known as “ see 
and avoid ' ’, has become a significant challenge to the routine operation of unmanned aircraft 
systems (UAS) within civil airspace. Until UAS developers can demonstrate an equivalent 
capability to sense and avoid other aircraft, UAS flight operations will be restricted to ensure the 
safety of all aircraft operating in the National Airspace System (NAS). 

This paper presents a definition of equivalent level of safety (ELOS) for UAS, as it pertains to 
see and avoid. The scope of this paper is further limited by only addressing the aircraft to 
aircraft aspect of collision avoidance. Future efforts will address collisions on the surface and 
other situations. The definition described in this paper is based on the functions necessary to 
sense and avoid traffic that represents a mid-air collision risk. The functions are independent of 
any particular solution or technology, to allow developers of UAS the ability to determine a 
sense and avoid solution for their system, considering its’ performance characteristics, level of 
autonomy, and type of airspace intended for UAS operations. 

See and avoid is one element of the conflict management process, which addresses the risk of 
collision between aircraft and hazards. The requirement for see and avoid is contained in Title 14 
Code of Federal Regulations Part 91, section 91.113. Without a pilot on-board an UA, an 
alternative method of compliance with that requirement is necessary, and an equivalent level of 
safety finding is therefore required for UAS type certification. This means that unmanned aircraft 
must achieve the same or less risk of collisions as compared to manned aircraft, in order to meet 
the see and avoid requirement. 

The sense and avoid EFOS requirement for UAS is stated as follows: 

The sense and avoid capability of the UAS shall meet the requirements of 14 CFR 

91.113, by providing situation awareness with adequate time to detect conflicting 

traffic and take appropriate action to avoid collisions. 

To be useful, this sense and avoid requirement needs to be further defined in terms of functional 
capabilities and key parameters. The basic functional capabilities of any sense and avoid system 
are surveillance, avoidance, and system quality. The surveillance and avoidance capabilities are 
described through functions involving scanning the surrounding airspace for other aircraft, 
determining if the detected aircraft is on a collision trajectory, deciding what evasive maneuver 
must be performed if any, and initiating the appropriate maneuver allowing sufficient time for 
the pilot’s own aircraft to respond and maintain separation. 

The surveillance capability is further defined by the key parameters of minimum detect time, 
field-of-regard, and track capability. These parameters define the surveillance volume of the 
sensing system(s), and description of the ability to provide situation awareness around the 
unmanned aircraft. 

The avoidance capability is further defined by the key parameters of minimum miss distance, 
miss distance probability, and system reaction time. These parameters address the system 
collision definition, and the ability of the system to ensure avoidance of traffic in both spatial and 
time frames of reference. 
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The key parameters for system quality address the integrity, continuity, and interoperability of 
the UAS system for sense-and-avoid. While the surveillance and avoidance capabilities address 
the capability of the system when it is working, the system quality parameters address the ability 
of the system to be available when needed, provide valid inputs, and work well with other 
subsystems in the UAS and other systems/users in the NAS. 

These key parameters will provide designers with the necessary standards/specifications to build 
to; and will provide regulators with the necessary standards/specifications to evaluate against. 
These are the minimum parameters necessary to describe sense-and-avoid system performance 
for ELOS consideration. These measures must be used in aggregate to properly define sense and 
avoid ELOS for a UAS and associated subsystems. The exclusion of any one measure will result 
in an incomplete or very limited sense and avoid ELOS definition. There is no priority order or 
ranking within the measures. 

This sense and avoid ELOS definition is not specific to any one particular solution or 
technology. The definition provides an approach for UAS manufacturers and the FAA to employ 
when developing and/or evaluating UAS sense and avoid capability and subsystems. It is the 
overall project’s desire that the content found within this report be referenced and used in 
establishing the necessary rules, regulations, and policies for UAS regarding sense and avoid. 
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1.0 


INTRODUCTION 


1.1 Purpose 

The ability of a pilot to detect other aircraft and maintain separation has been the foundation of 
safe aircraft operations since the beginning of aviation. This process, most commonly known as 
“ see and avoid ' ’, has become a significant challenge to the routine operation of unmanned aircraft 
systems (UAS) within civil airspace. The fact that UAS do not carry a pilot to perform this 
function has been the cause for many to speculate that this will compromise the safety of other 
airspace users. Until UAS developers can demonstrate an equivalent capability to sense and avoid 
other aircraft, Federal Aviation Administration (FAA) regulations will continue to place 
restrictions on UAS flight operations relative to this requirement, to ensure the safety of all aircraft 
operating in the National Airspace System (NAS). The purpose of this paper, therefore, is to 
present a recommended approach for defining an equivalent level of safety (ELOS), as it 
pertains to see and avoid. 

1.2 Scope 

The scope of this paper is limited to the sense and avoid aspect of ELOS. ELOS is much broader 
than just the sense and avoid topic, and encompasses almost all aspects of a systems operations. 
However, other safety-related aspects of ELOS that are unrelated to sense and avoid, are reserved 
for future study. This paper is further limited in scope by only addressing the aircraft to aircraft 
aspect of collision avoidance and leaves the analysis of collisions on the surface and other 
situations to future efforts. 

The approach described in this paper is based on the functions necessary to sense, and avoid traffic 
that represents a mid-air collision risk and is deliberately independent of any particular solution or 
technology. This independence from any particular technology should therefore allow developers 
of UAS platforms the ability to determine a final solution unique to their UAS’s performance 
characteristics, level of autonomy, and type of airspace intended for UAS operations. 

1.3 Project Sponsor and Participants 

The project sponsor for this effort is the National Aeronautics and Space Administration (NASA) 
Access 5 Project Office located at NASA Dryden Flight Research Center in Edwards, CA. This 
paper was developed by the Access 5 Policy and Cooperative Conflict Avoidance (CCA) Work 
Packages, comprised of individuals from Northrop Grumman, Lockheed Martin, General 
Atomics/ Aeronautical Systems, NASA Langley, and Modern Technology Solutions, Inc. 

In addition to the above participants, the information contained within this report has been argued 
and discussed at numerous workshops containing members from industry, government, and 
academia. The comments and recommendations received during each of these workshops have 
also helped to shape the current version of this paper. 
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1.4 NASA Access 5 Project 

Access 5 is a national project sponsored by NASA, with participation by the FAA advisors, DoD, 
and industry, to introduce civil High Altitude Long Endurance (HALE) UAS to routine flight in 
the NAS. Access 5 commenced in May 2004 and is slated to run for five or more years. While 
Access 5 is limited to HALE it is intended that this document be developed such that it has general 
applicability to airspace access and operations for UAS. 

The goal of Access 5 is to assist in the development of policies and procedures, demonstrate the 
enabling technologies, and identify infrastructure to promote a robust civil market for HALE 
UAS. Access 5 will address UAS airworthiness certification, flight operations, and crew 
certification. Project efforts will also include the development of appropriate standards, working 
where appropriate, through existing national standards groups. The project products are policy and 
procedure recommendations on UAS system airworthiness certification, UAS flight operations, 
UAS pilot certification, and appropriate standards. The project will identify mature technologies in 
several areas, including conflict avoidance and communications, and will also provide 
recommendations on maintenance for continued airworthiness, currency for pilots, and 
guidelines/processes for safe operation. 

Access 5 plans call for integrating HALE UAS into the NAS through a four-step process: 

Stepl - Routine operations of HALE UAS above Flight Level (FL) 400 (40,000 feet) 
with takeoff and landing within pre-coordinated/restricted airspace. Plus, 
proposals/guidance for an initial level of certification (experimental 
airworthiness) will be developed. 

Step 2 - Routine operations above FL180 (18,000 feet) with takeoff and landing 
within pre-coordinated/restricted airspace. A type certification basis for 
HALE UAS will also be developed. 

Step 3 - Routine operations above FL180 and access to UAS designated airports with 
emergency landings in restricted areas. In addition, the guidance/proposals for 
restricted airworthiness certification will be developed. 

Step 4 - Routine operations above FL180 and access to UAS designated airports, 
including emergency landings (i.e., true "file-and-fly"). The primary 
difference in operations between Steps 3 and 4 from an operating perspective, 
is the ability to handle emergencies or contingency landings in non-restricted 
airspace. In addition, the guidance/proposals for standard airworthiness 
certification of HALE UAS will be developed, along with guidance for an air 
operations certificate. 
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2.0 


BACKGROUND INFORMATION 


2.1 Current Operational Environment for UAS 

The demand to operate HALE UAS within the NAS is expected to increase significantly in the 
near future. This demand is being fueled by federal, non-DOD markets and emerging civil 
opportunities. The growth of the HALE UAS industry requires technology standards and 
regulatory criteria (certification of aircraft and the individuals who fly them) and operating 
standards (flight rules) that provide for the safe integration of this new aviation technology into the 
mainstream of airspace users. Because there currently does not exist any regulatory or procedural 
guidance for unmanned aircraft, the governing FAA authorities directly manage the ability to 
operate UAS within the NAS. Since most UAS are currently operated by the military, operations 
within civil airspace are treated as exceptional, one time events and authorization to fly is granted 
on a per mission basis in accordance with procedures contained within FAA Order 7610.4K [1], 
This FAA Order requires that the UAS operator apply for a Certificate of Authorization (COA) at 
least 60 days prior to the proposed commencement date. 

Under the COA process, UAS are allowed to operate within the NAS, but only through means of 
segregation. Even with a valid COA, UAS are unable to routinely operate with the same 
flexibility as manned aircraft and are subject to numerous restrictions prohibiting simple activities 
such as operating above populated areas of the country or taking off and landing outside special 
use airspace (SUA). Regardless of the mission or role, UAS are restricted as to how, when, and 
where they can operate. 

By not having routine access to the NAS, the civil and commercial viability for UAS is severely 
limited. The manufacturers often have difficulty in attracting investment capital, as well as 
difficulty in obtaining flight insurance; both of which lead to high operating costs. The UAS 
industry must take several steps forward to become more attractive to the civil and military sector. 
Routine operations within civil airspace are essential to their ability to thrive in the marketplace, 
both domestically and internationally. 

2.2 Current Process used for Preventing Mid-Air Collisions 

According to 14CFR 91.113, regardless of whether an aircraft is operating under visual flight rules 
(YFR) or instrument flight rules (IFR) the pilot in command is to remain vigilant to see and avoid 
other aircraft [2], Although this responsibility ultimately rests upon the pilot’s shoulders, other 
factors assist him/her in safely performing this task. To increase aviation safety, the FAA has 
established an integrated, overlapping process intended to minimize the risk of mid-air collisions. 
Figure 1 depicts the layered approach that is currently in place for manned aviation and what the 
UAS industry should adopt if they desire to integrate into the NAS. 
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Airspace Procedures 

The first line of defense in the prevention of mid- 
air collisions are existing rules, regulations, and 
procedures. If adhered to, these procedures should 
prevent most accident scenarios from ever 
occurring. 

Air Traffic Management 

The second line of defense is the services provided 
by Air Traffic Management (ATM). If an aircraft 
is under positive control, it will receive separation 
services from ATM for other participating aircraft 
within the surrounding airspace. The air traffic 
authority assumes responsibility for providing 
separation for all IFR traffic and directs the 
participating aircraft to either maintain or change 
their flight path, including providing traffic 
advisories on other aircraft, time permitting. 

See and Avoid / Cooperative Traffic Avoidance 
See and avoid is the last line of defense used by Figure 1: Current Process for Avoiding 
pilots to avoid collisions with other aircraft and Mid-Air Conflicts/Collisions 

obstacles located in the air and on the surface. The 

use of on-board traffic advisory systems to provide situational awareness concerning all 
cooperative traffic within a certain range and altitude, can enhance the ability of the human pilot 
inside the cockpit to detect and track potential collision threats. These systems, mandatory on all 
aircraft that carry 10 or more passengers, also provide a traffic advisory to the pilot of an 
impending collision and some can even provide a resolution advisory directing an avoidance 
maneuver. While technology such as cooperative traffic avoidance systems can cue a pilot to the 
presence of conflicting traffic, the use of human eyesight to see-and-avoid airborne hazards, has 
been the ultimate collision avoidance tool contributing to safe flight since the beginning of 
aviation. The discussions contained within this paper focus on on-board and/or off-board sensing 
means and aircraft control required for UAS to provide the airborne aircraft to aircraft segment of 
this layer of collision avoidance without an on-board pilot. The discussions assume that this 
collision avoidance capability may be provided for both cooperative or non-cooperative traffic 
(see section 2.3.2). 

Conflict Avoidance vs Collision Avoidance 

The overall process described above is one of conflict management, which according to RTCA, is 
the limiting of the risk of collision between aircraft and hazards, to an acceptable level. However, 
the distinction needs to be made between avoiding conflicts with other traffic as opposed to 
avoiding collisions with other traffic (Figure 2). A conflict is any situation involving an aircraft 
and hazard in which the applicable separation minima may be compromised. A collision is a 
situation involving the physical contact between an aircraft and an airborne hazard. Conflict 
management involves all of the layers shown in Figure 1, including collision avoidance. Collision 
avoidance is the last layer of conflict management, and must activate when the separation mode 
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has been compromised. Airspace procedures and ATM represent the separation layers of conflict 
management. Collision avoidance is not part of separation provisions, and collision avoidance 
systems are not included in determining the calculated level of safety required for separation 
provision [ 34 ]. Sense and avoid is therefore part of collision avoidance as distinguished from 
maintaining separation minima for conflict avoidance. This distinction and focus on collision 
avoidance will be important as the requirement for sense and avoid safety equivalence is 
developed. 



Figure 2: Collision Avoidance versus Conflict Avoidance 


2.3 Collision Avoidance Challenges 

There are several external elements that make collision avoidance challenging. Some of these 
elements include: various classes of airspace, different types of aircraft, and environmental 
conditions; as well as independent maneuvers of aircraft, competition for resources (airport, VOR, 
etc.), and ability to detect other aircraft. Each of these will be briefly discussed below. 


2.3.1 Types of Airspace 
Operations 

The first of these elements is the 
type of airspace and existing 
manned operations where the UAS 
will be operating. There are several 
different classes of airspace through 
which the UAS may have to 
transition in order to accomplish the 
desired mission. Within each of 
these classes of airspace (designated 
Class A - Class G), the FAA has 
established specific guidelines and 
constraints to reduce the collision 



Figure 3: Airspace Example 




potential. For example, traffic above FL180 (18,000 ft) is required to be under FAA positive 
control and must abide by instrument flight rules (IFR) for that airspace. With respect to conflict 
avoidance, air traffic control (ATC) accepts the responsibility for providing separation between 
aircraft operating at these altitudes. Actually, all IFR aircraft are normally required to use an 
approved transponder, regardless of the airspace they are operating in, which enables the FAA to 
track the aircraft and provide traffic separation information to the pilot. Traffic maneuvers above 
FL180 also tend to be more constrained than maneuvers at lower altitudes. Aircraft operating 
below FL180 may or may not be under ATC control. In fact, at these lower altitudes there may be 
both ATC controlled and un-controlled air traffic transitioning within the same class of airspace 
with varying surveillance avionics equipage. This mix of aircraft type and capabilities can create a 
challenging operational environment for an unmanned aircraft to transition. Figure 3 shows a 
depiction of the many different classes of airspace found within the NAS. 

2.3.2 Cooperative vs. Non-cooperative Aircraft 

Essentially, two types of traffic must be detected for collision avoidance: cooperative and non- 
cooperative. Cooperative aircraft, which comprise most of the total air traffic, are those aircraft 
that have an onboard transponder or other means to “cooperate” with normal air traffic 
surveillance systems. One of the most common situational awareness tools used to detect and 
locate cooperative aircraft is the Traffic Alert and Collision Avoidance System (TCAS), required 
on all aircraft in the United States capable of seating 10 or more passengers. The transponder 
signal from a cooperative aircraft is received by other aircraft possessing the TCAS equipment and 
is used to display the cooperative aircraft’s position and altitude information. Non-cooperative 
aircraft do not “cooperate” with air traffic surveillance systems and therefore need to be detected 
through alternative surveillance means such as human vision for see and avoid. Some small 
general-aviation airplanes, gliders, and balloons, with very few exceptions, fall into this category 
of traffic. Therefore, a truly robust sense and avoid system must be capable of detecting both non- 
cooperative and cooperative aircraft to avoid a potential collision. Candidate on-board 
technologies capable of detecting non-cooperative objects include electro-optical (infrared or 
visible), radar, and ladar. Off-board technologies such as broadcasting FAA primary or secondary 
radar information may also be useful in detecting non-cooperative objects. 

2.3.3 Detection Characteristics 

The ability to detect other aircraft largely depends upon the relative size, background contrast, 
structural materials, propulsion system, and intercept geometry. These characteristics can 
obviously present challenges to different sensor technologies that may be employed for such an 
application. For example, a visible sensor like the human eye can detect a brilliant orange colored 
aircraft much better that it could detect a camouflage one. Radar, or other radio frequency (RF) 
type sensors on the other hand, would not differentiate between aircraft coloring schemes. The 
material with which an aircraft is made can also drastically affect how easily it may be detected 
using RF and infra-red (IR) sensors. For example, cloth and composite aircraft are much more 
difficult to detect with radar than a metal aircraft would be; while an infrared camera could detect 
a hot-air balloon much easier than radar could. Similarly, the type of propulsion system (none, 
single prop, dual prop, jet engine) used by an aircraft can affect how easily it can be detected. 
These different propulsion types can have a wide range of radar cross-sections and heat signatures. 
The final aircraft characteristic to be mentioned is that of aircraft aspect. The geometry, at which 
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the intruder aircraft is approaching, can also make detections more or less difficult because of the 
cross-section of the aircraft presented to the pilot or sensor. 


2.3.4 Weather and Environment 

The last challenge affecting collision avoidance is the environmental conditions in which an 
aircraft will be operating. Aircraft come in many different shapes and sizes and are capable of 
flying at a wide range of altitudes. Some are designed to fly along the nap of the earth, while 
others travel at very high altitudes. Depending upon the UAS operational environment, the sense 
and avoid sensor may need to operate under a variety of altitudes, temperatures, pressures, and 
forms of precipitation (snow, rain, mist, or fog). 

A sensor’s performance may also depend upon when it is used, as well as where it is used. If an 
aircraft is intended for both day and night operations, the collision avoidance system should have 
limited degradation resulting from the ambient lighting conditions. Furthermore, the ability to 
detect conflicting aircraft will vary significantly with the object’s background. For example, an 
aircraft with the blue sky as a background can be much easier for an electro-optic sensor to detect 
compared to an aircraft with ground clutter as its background. 


7 



3.0 UNDERSTANDING ELOS 


3.1 The Basis for an ELOS Requirement 

UAS intending to routinely operate within the NAS must demonstrate that they can meet the 
existing rules and regulations currently in place for manned aircraft. If any of these standards 
cannot be met, then the shortcoming must be addressed using the procedures established by the 
FAA for meeting such deficiencies. 

One of the precedents for having to meet an equivalent level of safety for routine operations 
within the NAS is found in 14CFR 21.21, paragraph (b) [3]. This regulation states that an 
applicant is entitled to a type certificate for an aircraft, if the type design, test reports, and 
computations necessary to show that the product to be certificated meets the applicable 
airworthiness requirements of the Federal Aviation Regulations. However, upon examination of 
this information, if any airworthiness provisions are not complied with then they must be 
“. . .compensated for by factors that provide an equivalent level of safety ” [3], 

The FAA process for demonstrating that a deficiency meets an equivalent level of safety is 
identified in Paragraph (g.) of FAA Order 8110.4B [4J. This order identifies the responsibilities 
and procedures for FAA aircraft certification personnel responsible for the certification process 
required by Title 14 of the CFR for civil aircraft, aircraft engines, and propellers necessary before 
granting a Type Certificate. When required, the basis may include an equivalent level of safety 
finding when literal compliance with a certification regulation cannot be met and compensating 
factors exist that can be shown to provide an equivalent level of safety. 1 

3.2 Need for UAS to Meet an ELOS for See and Avoid 

Not only is see and avoid a vital element of safe aircraft operations, but it is also required by Title 
14 of the Code of Federal Regulations (CFR). Paragraph 91.113 (b) states: 

14 CFR 91.113(b) General. When weather conditions permit, regardless of 
whether an operation is conducted under instrument flight rules or visual flight 
rules, vigilance shall be maintained by each person operating an aircraft so as to 


[ FAA Order 8110.4B; Paragraph (g.) Equivalent Level of Safety Finding 

(1) Equivalent level of safety findings are made when literal compliance with a certification regulation cannot be 
shown and compensating factors exist which can be shown to provide an equivalent level of safety (reference § 
21 .21(b)(1) and Order 8100.5, paragraph 408.) 

(2) The applicant submits to the ACO the proposed equivalent level of safety. The ACO then submits to the 
directorate the proposed equivalent level of safety with recommendations. The accountable directorate makes all 
equivalent level of safety findings. 

(3) In documenting an equivalent level of safety: 

(a) List the applicable regulation; 

(b) Describe the features of the design that require the equivalent level of safety findings; 

(c) Describe any design changes, limitations, or equipment imposed to make the equivalency; and 

(d) Provide an explanation of how the actions taken provide an equivalent level of safety to that intended by the 
regulation. 

(4) All equivalent level of safety findings must be listed on the TCDS or on the STC. 
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see and avoid other aircraft. When a rule of this section gives another aircraft the 
right-of-way, the pilot shall give way to that aircraft and may not pass over, 
under, or ahead of it unless well clear. 

Part 91 of Title 14 CFR specifically identifies 14 CFR 91.1 13 as an operating regulation. Without 
a pilot on-board an UA, an alternative method of compliance with that requirement is necessary, 
and an equivalent level of safety finding is therefore required for a UAS type certification process. 
Developers of unmanned aircraft intended to operate in the NAS, must have an approach that can 
be used to demonstrate to the appropriate regulatory agencies that the UAS system design meets 
an equivalent level of safety to that of a manned aircraft. 

A similar requirement can be found in Section 12-9-2, of FAA Order 7610.4K, Special Military 
Operations [1], which addresses the operation of military UAS outside Restricted Areas and 
Warning Areas. Sub-paragraph a.4 states that all requests to operate UAS must provide the FAA 
with the “method of pilotage and proposed method to avoid other traffic.” In a note, this sub- 
paragraph goes on to state: 

Approvals for UAS operations should require the proponent to provide the UAS 
with a method that provides an equivalent level of safety, comparable to see-and- 
avoid requirements for manned aircraft. Methods to consider include, but are not 
limited to; radar observation, forward or side looking cameras, electronic detection 
systems, visual observation from one or more ground sites, monitored by patrol or 
chase aircraft, or a combination thereof. 

Although the ELOS phrase found in FAA Order 7610.4K is quite vague, it can be used as a 
starting point for exploring civil UAS type certification. To do this however, it is incumbent upon 
the UAS industry and the FAA to come to a consensus approach for demonstrating ELOS, which 
must include an acceptable definition of ELOS for sense and avoid. Ultimately, this approach 
should be sufficient enough to provide industry with a set of functional capability requirements 
that could be used to derive sense and avoid subfunction performance requirements for UAS 
collision avoidance systems. The FAA could also use this established approach and definition as a 
means to derive the certification requirements for such a system. 

3.3 Dissecting the See and Avoid ELOS Phrase 

Before an attempt is made to define ELOS, it is beneficial to first dissect the phrase into its three 
key elements: (1) equivalence, (2) safety, and (3) see and avoid requirements for manned aircraft. 
Each of these elements will be discussed in the following paragraphs to ensure a common 
understanding before addressing the definition of the phrase as a whole. 

3.3.1 Equivalence 

Equivalence, simply defined, means “same.” So the question becomes one of “the same as what?” 
Equivalence should be with respect to the desired end result of see and avoid based collision 
avoidance and not the individual processes leading up to that result. The see and avoid 
responsibility of the pilot is an important element of collision avoidance, but primarily provides a 
last line of defense if airspace procedures, air traffic management, and cooperative traffic 
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avoidance measures break down or are not available. See and avoid limitations placed on a pilot 
by workload, cockpit design, human vision, and other factors all suggest that equivalence should 
not be applied directly to the processes that a pilot performs when avoiding another aircraft. For 
example, a see and avoid system designed to mimic a human pilot would need to stop detecting 
for several seconds every few minutes to account for the time that a human pilot must routinely 
check the cockpit avionics instruments and flight map. It might be reasonable to expect an actual 
improvement in detection performance over a human pilot’s ability to spot other traffic since 
electronic sense and avoid systems are not distracted, fatigued, or affected by physiological 
limitations. Equivalence should therefore be applied to a sense and avoid system’s ability to 
permit the successful completion of a collision avoidance maneuver when required. 

3.3.2 Safety; as it Applies to See and Avoid 

UAS operations must be shown to be “safe” if they are to be successfully integrated into the NAS. 
This will require those who design, build, operate, and maintain UAS to convince the regulators 
and the public that UAS are “safe” [5], In this context, “safe” represents an acceptable level of 
risk of damage or injury to persons or property in the air or on the ground. Some might suggest 
that the UAS system be “absolutely safe”. However, this implies zero risk and is therefore an 
unobtainable state. An alternative definition for safe is “acceptably safe”. The question then 
becomes: what is an acceptable level of safety or risk. 

Historically, manned aviation has achieved a relatively high level of safety. With this threshold 
and the requirement of transparence in mind, UAS operations must be as safe as manned aircraft 
operations insofar as they must not present or create a hazard to persons or property in the air or 
on the ground. The methods required by the civil aircraft certification process have played a major 
role in achieving the high safety levels evident in aviation [5], There is, however, a distinction 
that must be made concerning overall system safety and safety as it pertains to see and avoid only. 
As mentioned in Section 1.0, this paper is intended to define the “equivalent level of safety, 
comparable to see and avoid” as opposed to the much broader topic of “equivalent level of safety 
to that of a manned aircraft”. This document focuses on the level of safety value for see and avoid 
only. This level of safety should support the overall ELOS value that will be used for the UAS 
system. The remainder of this document will discuss where this line for collision avoidance safety 
gets drawn. 

3.3.3 See and Avoid Requirements for Manned Aircraft 

With respect to see and avoid, the foremost requirement contained within the Federal Aviation 
Regulations (FAR) is 14 CFR Part 91.113, entitled Right of Wav Rules: Except Water Operations 
[2], Sub-paragraph (b) states: “ When weather conditions permit, regardless of whether an 
operation is conducted under instrument flight rules or visual flight rules, vigilance shall be 
maintained by each person operating an aircraft so as to see and avoid other aircraft. ” 

Similarly, a requirement within ICAO regulation is Annex 2, entitled Rules of the Air [6], 
Section 3.2 of this annex, which is entitled Avoidance of Collision, states: “I t is important that 
vigilance for the purpose of detecting potential collisions be not relaxed on board an aircraft in 
flight, regardless of the type of flight or the class of airspace in which the aircraft is operating, 
and while operating on the movement area of an aerodrome. ” 
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It should be clear from these references that the operator cannot rely solely upon ATC for 
providing separation if weather permits visual detection of conflicting traffic. Since UAS do not 
possess a pair of human eyes onboard the aircraft, some other means for providing a sense and 
avoid capability is necessary. This capability, regardless of what it is, should be able to achieve an 
“equivalent level of safety, comparable to see and avoid requirements for manned aircraft”, so as 
not to pose a hazard to persons or property, both in the air and on the surface. 

The term “sense,” as in “sense and avoid,” will be used to denote detection capability provided by 
means other than human vision. The term “see” will be used when referring to human vision. 

In summary, the phrase “equivalent level of safety, comparable to see and avoid requirements for 
manned aircraft,” means that unmanned aircraft must achieve the same or less risk of collisions as 
compared to manned aircraft, in order to meet the Federal Aviation Requirements concerning see 
and avoid (14 CFR 91.113). 
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4.0 APPROACH FOR SENSE AND AVOID ELOS 


4.1 Approaches for Defining Sense and Avoid ELOS 

There are two general approaches that may be used to determine whether an unmanned aircraft is 
as safe as a manned aircraft. The first approach compares the statistics of a manned system to that 
of the unmanned system, whereas the second approach compares the performance of the manned 
system and the FAA regulatory requirements, to that of the unmanned system. While both 
approaches can provide meaningful insight, only the performance approach defines ELOS using 
leading indicators to measure UAS capability. The statistics approach, on the other hand, relies 
upon lagging indicators that can only be validated after a sufficient amount of UAS operations 
statistics are collected. Although statistics by definition are always lagging indicators they can still 
be an important element of identifying a need for tradeoffs between the performance-based 
parameters, if required, in order to ensure routine access to the NAS. 

Both the statistical and performance-based approaches have a variety of different methods that 
could be used to achieve an ELOS definition for sense and avoid. A detailed description of each 
potential approach and methodology, including their strengths and weaknesses, is contained in 
Appendix A at the end of this paper. 

4.2 Recommended Approach for Defining Sense and Avoid ELOS 

The suggested approach for defining sense and avoid ELOS for sense and avoid actually combines 
portions of the statistical and performance based approaches, which are discussed in detail within 

Appendix A. The sense and avoid ELOS definition (Section 5) should be used as basis for 
establishing the leading indicators needed to gain initial access to the NAS for UAS, while 
the Target Level of Safety Method should be used to establish the top-level measure of 
effectiveness. 

This approach establishes a set of well-defined, measurable parameters that can be used to 
evaluate a sense and avoid system design through simulation, analysis, demonstration, and test. 
Using a performance-based approach will result in a greater level of confidence that a proposed 
sense and avoid design meets the necessary safety criteria. These parameters will provide the FAA 
with a “ leading indication ” that the UAS system is capable of maintaining safe separation and is 
suitable to fly within civil airspace. This should therefore also result in a higher probability of 
approval by the regulatory agencies responsible for ensuring the safety of the NAS. 

Even though the performance-based approach is proposed for creating the initial ELOS definition 
and gaining approval to operate, the statistical approach still plays an important role in this 
combined approach. A certain amount of simulations and analysis should also be used to show 
how the sense and avoid system will be able to achieve a high degree of reliability and safety, 
necessary to achieve ELOS to that of a manned aircraft. If problems are identified, changes to the 
performance-based approach would then be required to ensure that only UAS with an equivalent 
level of safety were gaining approval to operate. 
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5.0 DEFINING SENSE AND AVOID ELOS 

It is important to recognize that a “one size fits all” approach to performance of a UAS sense and 
avoid system may not be possible, nor desirable, since UAS operations can vary widely. For 
instance, while both a high-speed, high-altitude and low-speed, low-altitude UAS must provide an 
equivalent level of safety for see-and-avoid, the sense and avoid system used in each aircraft may 
be different. The selected performance parameters should be allowed to be different, to account for 
the environments in which each system operates. The definition of ELOS for sense and avoid will 
start with the basic requirement followed by the fundamental capabilities, and finally a breakout of 
the measures or parameters that describe the attributes of the system will complete the definition. 
The measures should represent all of the critical elements which define any sense and avoid 
system’s ability to provide an ELOS for sense and avoid. This sense and avoid ELOS definition 
should adequately address cooperative traffic, non-cooperative traffic, autonomous and non- 
autonomous situations and implementation. In addition, the definition should account for UA 
maneuvers as well as straight and level situations. 

5.1 Sense and Avoid ELOS Requirement 

The sense and avoid ELOS requirement as stated in Section 3, is based on 14 CFR 91.113 “ Right- 
of-way rules: Except water operations ”, which is the genesis of the “see and avoid” requirement. 
Using this definition as the starting point, the sense and avoid ELOS requirement for UAS is as 
follows: 

The sense and avoid capability of the UAS shall meet the requirements of 14 CFR 
91.1 13, by providing situation awareness with adequate time to detect conflicting traffic 
and take appropriate action to avoid collisions. 

5.2 Sense and Avoid ELOS System Capabilities 

Although the basic goal is to define an equivalent level of safety to that of a manned aircraft, it 
would be unreasonable to establish sense and avoid system requirements based solely upon human 
parameters. In addition to ensuring that collision avoidance requirements are adequate for UAS to 
operate routinely in civil airspace, the requirements must not be overly burdensome. Since UAS 
have a wide variety of performance characteristics, the values for each performance parameter 
could vary substantially between the various UAS types. What is essential for a high performance 
UAS may be unnecessary and impractical for a less capable UAS. Any requirement more 
stringent than one that is derived from the existing aviation regulations should be considered as 
exceeding “an equivalent level of safety, comparable to see-and-avoid requirements for manned 
aircraft”. If a more stringent requirement is to be imposed on UAS, the rationale for the additional 
capability, exceeding that required for manned aviation, should first be substantiated. 

Once see and avoid ELOS has been defined, requirements can then be proposed for collision 
avoidance systems. These parameters will provide designers with the necessary 
standards/specifications to build to; and will provide regulators with the necessary 
standards/specifications to evaluate against. Before discussing any specific numbers or values, 
these parameters and appropriate Measures of Effectiveness (MOE) to evaluate a system against 
those parameters must first be determined. After the MOEs are determined, then the appropriate 
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measures of performance (MOP) may be determined, if necessary, to establish collision avoidance 
standards. 

Regardless of the technology selected for implementation of a collision avoidance solution, it 
should be able to meet certain functional capabilities that are measurable and together provide the 
required goal of achieving an equivalent level of safety. 

The six sense-and-avoid functions listed below were derived from a human model for mid-air 
collision avoidance, but they are applicable to any collision avoidance system. These functions 
involve scanning the surrounding airspace for other aircraft, determining if the detected aircraft is 
on a collision trajectory, deciding what evasive maneuver must be performed, if any, and initiating 
the appropriate maneuver allowing sufficient time for the pilot’s own aircraft to respond and 
maintain separation. To avoid collisions, these six essential functions must be performed by a 
UAS collision avoidance system. It is also important to note that these functions may occur within 
the aircraft element, control element, or both. The six sense-and-avoid functions are shown in 
Figure 4. 
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FI - Detect traffic 
F2 - Track traffic 

F3 — Identify the likelihood of a collision 

with one or more of the traffic 

F4 — Determine an avoidance 
maneuver 

F5 — Initiate avoidance maneuver 
F6 — Assess maneuver adequacy and 
complete the maneuver 


Figure 4: Sense-and- Avoid Functions 


These functions support the top-level function of a sense and avoid system to meet the 
requirements of 14 CFR 91.113 by providing situational awareness with adequate time for the 
UAS system to detect conflicting traffic and take appropriate action to avoid mid-air collisions and 
provide a framework for defining measures that can be used to determine the adequacy of a system 
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design. The sense and avoid requirement contains several important words/phrases which outline 
some of the capabilities of any system to meet the requirement. The following paragraphs expand 
on the key capabilities of a sense and avoid system for UAS. 

5.2.1 Surveillance Capability 

The phrase “situation awareness” in the sense and avoid requirement, implies a surveillance 
capability to detect and track traffic with sufficient resolution to give dynamic perspective to the 
traffic picture. The dynamic aspect of the situation awareness is necessary to present an accurate 
picture of the traffic flows, especially in situations where the UAS or the traffic, or both are 
maneuvering. This surveillance capability can be defined with the functions of detect and track 
(FI and F2 above). The detect function can be defined using the measures of minimum detect time 
and field of regard of the system. The track function can be defined using the measures of field of 
regard and track capability. Each of these measures are defined below. 

5.2.1. 1 Minimum Detect Time 

Minimum detect time is defined as the time from detection of an intruder until the completion of 
an evasion maneuver (functions F2 through F6 in section 5.2). In other words, the system must 
provide sufficient time after intruder detection for performance of all remaining collision 
avoidance functions resulting in successful execution of an avoidance maneuver if necessary. 

MDT >F2+F3+F4+F5+F6 


A system meeting this requirement is shown in Figure 4. 


Minimum Detect Time (MDT) 



Figure 5: Minimum Detect Time Definition 


The greatest distance that traffic would need to be detected would be in a head-on collision at 
maximum speed. In order for minimum detect time to be realistic, a maximum closure speed 
between the UAS and traffic should be defined relative to the capability of the UAS and potential 
traffic in the operating environment. Minimum detect time in effect, provides the range or depth 
aspect of the surveillance volume. 
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5.2.1.2 Field of Regard 

Field of Regard (FOR) is the azimuth and elevation 
aspects of the surveillance volume within which the 
collision avoidance system is able to detect and track 
other aircraft (Figure 6). The elevation limits should be 
broad enough to include climbing and descending 
traffic. The azimuth limits should be wide enough to 
detect converging traffic, and to clear the area ahead of 
a turn. The field of regard volume is centered on the UA 
flight path vector and aligned along the horizon. 



Figure 6: Field of Regard 


5.2.1.3 Track Capability 

Track capability addresses the track function. This measure 
includes track probability, probability of false alarm, and the 
total number of intruder aircraft tracks that the sensor is 
capable of simultaneously tracking and managing. Figure 7 
shows an example of a display providing the track of several 
traffic objects. 


5.2.2 Avoidance Capability 

The phrase “adequate time” in the sense and avoid 
requirement, means there is a critical time element in the 
definition. The phrase “appropriate action” implies that the 
system, including the pilot or autonomously, must be able to 
to provide the avoidance of a collision along with the correct right-of-way in a particular situation. 
Finally, the phrase “avoid collisions” means the focus is on avoiding a collision with the 
conflicting traffic versus avoiding a conflict with the traffic. The traffic avoidance capability can 
be defined using the four remaining functions - identify conflicting traffic, determine avoidance 
action, initiate avoidance maneuvers, and finally, assess and complete the avoidance maneuver. 
The avoidance functions can be defined by using the track capability measures under surveillance, 
along with a definition of a minimum miss distance, the miss distance probability, and a system 
reaction time. 



Figure 7: Track Capability 


decide on the appropriate maneuver 
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5.2.2.1 Minimum Miss Distance 


Minimum 

Miss ~<| 
Distance 


Minimum miss distance (MMD) addresses 
the assumption that any traffic flight path that 
intersects the UA flight path within a 
specified distance (horizontal and vertical) 
will result in a collision. The goal of the 
sense and avoid system is to prevent actual 
contact between the UA and any traffic. 

However, any practical implementation of a 
sense and avoid system will have to decide 
when a maneuver must occur to avoid a 
collision. This minimum miss distance or 
“collision definition” is a prerequisite for 
determining minimum detect time (described 
above) and miss distance probability 
(described below), and must be tailored to the 
specific UAS. Figure 8 provides an example 
of how a minimum miss distance might be 
calculated. In this example, the MMD is 
defined as the sum of one-half the wingspan 

of the traffic plus one-half the wingspan of the UA. Note that this is an example for only one 
dimension of the MMD. An actual MMD would be defined in three dimensions. 



Figure 8: Example Minimum Miss Distance 


5.2.2.2 Miss Distance Probability 

Miss distance probability (MDP) defines the probability that a UAS will avoid an intruding 
aircraft by the MMD. In order for a system with uncertainties to achieve a desired MDP, the UAS 
must have a target miss distance that is greater than MMD, as shown in Figure 9. 

For instance, a miss distance with a three sigma level probability would mean that the sense and 
avoid UAS system would have to plan to miss all traffic by a distance large enough to ensure that 
it would miss in 99.7% of collision encounters. A system with large uncertainties would be 
required to achieve a greater miss distance on average in order to ensure the required miss distance 
probability. Even a system with small uncertainties would still have a target miss distance greater 
than MMD. 



Figure 9: Miss Distance Probability Example Encounters 
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5.2.2.3 System Reaction Time 

System reaction time is measured from point at which a collision potential has been identified, to 
the completion of the avoidance maneuver (Figure 10). It is everything after detection and track, 
and includes the time to decide on an avoidance maneuver, plus the time to initiate and accomplish 
the avoidance maneuver. In a human based collision avoidance system, system reaction time 
includes the recognition time, decision time, psychomotor reaction time, aircraft control system 
time lags, and the time for the aircraft to move the minimum miss distance away from the collision 
point. System reaction time must be less than the minimum detect time plus track time. 

MDT > F2 + SRT 



SRT = F3 +F4+F5 + F6 



5.2.3 System Quality 

The surveillance and avoidance capabilities describe the performance capabilities of a sense and 
avoid system. In addition, there must be measures identified to describe how well the system 
performs its functions. The measures identified to define the sense and avoid system quality are 
miss distance probability, integrity, continuity, and interoperability. 

5.2.3. 1 System Integrity 

Integrity is defined as the ability of a sense and avoid system to identify degraded input data or 
system functionality and continue sense and avoid operations within the limitations of the 
degraded system [32]. Integrity includes the following lower-level measures of performance: fault 
tolerance (including software integrity management), and false maneuver rejection. It also 
includes the issuance of timely and valid warnings when the system cannot be reliably used. 

5.2.3.2 System Continuity 

System Continuity is a measure of the systems ability to complete a sense-and-avoid process once 
initiated [32]. This includes availability, reliability, and persistence. 

5.2.3.3 System Interoperability 

System interoperability is a measure of the level of degradation of other systems, including the 
National Airspace System (NAS) and the UAS, caused by sense and avoid system operation. This 
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includes nuisance to the NAS and operability, or ease of use for operators of the UAS. System 
interoperability also includes the ability of the UAS to provide right-of-way as necessary 
according to Federal Aviation Regulations. 

System interoperability also relates to disruption of the Air Traffic Control (ATC) System if the 
UAS sense and avoid system reacts early and maneuvers wide of potential conflicts. While 
operating under ATC, the regulations specify that a manned aircraft will not maneuver unless 
there is an absolute necessity. 

CFR 91.123(a) When an ATC clearance has been obtained, no pilot in command may deviate 
from that clearance unless an amended clearance is obtained, an emergency exists, or the 
deviation is in response to a traffic alert and collision avoidance system resolution advisory... 

Though not stated, there is an implication that failure to adhere to this principle will not just 
disrupt ATC, it will compromise the safety of the air traffic system. Early activations or alarms by 
a sense and avoid system could negate the instructions that ATC was about to issue and force him 
or her to reroute other traffic unnecessarily. Early sense and avoid activations could also conflict 
with resolution advisories issued by other aircraft TCAS. Therefore, in order for a sense and avoid 
system to achieve ELOS to a manned aircraft, it must not force a maneuver unless it is absolutely 
necessary. 

5.3 Sense and Avoid ELOS Definition 

Using the requirement for sense and avoid described above, and the measures of capability which 
describe any sense and avoid systems ability to provide an ELOS to manned aircraft, the definition 
of sense and avoid ELOS is therefore the following: 

Equivalent level of safety to manned aircraft see-and- avoid for UAS, is the providing of situation 
awareness with adequate time to detect conflicting traffic and take appropriate action to avoid 
collisions. This sense-and-avoid capability is described by measures addressing surveillance, 
avoidance, and system quality. The following measures describe the sense-and-avoid system 
capability: 

1 . Minimum Detect T ime 

2. Field-of-Regard 

3. Track Capability 

4. Minimum Miss Distance 

5. Miss Distance Probability 

6. System Reaction Time 

7. System Integrity 

8. System Continuity 

9. System Interoperability 

These are the minimum measures necessary to describe sense-and-avoid system performance for 
ELOS consideration. These measures must be used in aggregate to properly define sense and 
avoid ELOS for a UAS and sense and avoid subsystem. The exclusion of any one measure will 
result in an incomplete or very limited sense and avoid ELOS definition. There is no priority order 
or ranking within the measures. Additional measures for a specific system may also be used to 
provide a more complete description of a proposed implementation. 
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The definition presented here is qualitative in nature, in order to retain maximum design flexibility 
for sense and avoid system developers. The definition includes measures as leading indicators, to 
permit understanding and evaluation of proposed systems, in support of an overall requirement 
which can only be validated after the fact (a lagging indicator of performance). 
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6.0 SUMMARY 


The discussion of the sense and avoid requirement and the associated capabilities and measures 
provide the definition of what equivalent level of safety means for sense and avoid as it applies to 
UAS. The definition is qualitative in nature, in order to retain maximum design flexibility for 
sense and avoid system developers. 

UAS are currently allowed to operate in the NAS through a process of precoordination and 
segregation of UAS from manned aircraft. Defining an ELOS for sense and avoid for UAS would 
be a key factor in the eventual integration of UAS in the NAS. The current process for preventing 
mid-air collisions for manned aircraft in the NAS utilizes a layered approach consisting of both 
conflict avoidance and collision avoidance techniques. The conflict avoidance techniques for UAS 
will be identical to manned aircraft conflict avoidance, so the sense and avoid ELOS definition is 
focused on the collision avoidance portion of preventing mid-air collisions. 

The requirement for a sense and avoid capability is contained in 14 CFR 91.113. Since UA do not 
have a human pilot onboard, the UAS needs to provide other means to provide an ELOS to 
manned aircraft for the see and avoid capability. The ELOS phrase can be partitioned into its three 
key elements: 1) “equivalence”, 2) “level of safety”, and 3) “see-and avoid requirements for 
manned aircraft,” in an attempt to better understand the phrase as a whole. A combined approach 
has been developed for defining sense and avoid ELOS and for verifying the system is performing 
safely and reliably. The performance-based capability measures defined will provide “ leading 
indicators ” to describe to the FAA the systems ability to meet an equivalent level of safety for 
sense and avoid. These leading indicators could also be supported with a sufficient amount of 
simulation and analysis showing that a sense and avoid system, which contains the performance- 
based characteristics, can achieve the necessary safety level. Once the FAA has confidence that 
the proposed system will provide an ELOS and has permitted the system to operate within the 
NAS, the performance of this system should then be evaluated to ensure the overall reliability and 
safety numbers are being met. This later evaluation is what is referred to as the “ lagging 
indicator ” that an equivalent level of safety has been met. 

Throughout this paper, a conscious effort was made to define ELOS for sense and avoid in a 
logical order and in a manner that does not specify any one particular solution or technology over 
another. It is important to keep an adequate amount of flexibility for the developers of UAS 
platforms since the final solution could be unique to an UAS. The results address the phrase 
“equivalent level of safety” which has been impeding the UAS industry for several years, and 
provide an approach for UAS manufacturers and the FAA to employ when develop ing/evaluating 
UAS sense and avoid subsystems. It is the project’s desire that the content found within this report 
be referenced and used in establishing the necessary rules, regulations, and policies for UAS 
regarding sense and avoid. 
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7.0 ACRONYMS 


ADS-B 

Automatic Dependent Surveillance - Broadcast 

ATC 

Air Traffic Control 

CFR 

Code of Federal Regulations 

COA 

Certificate of Authorization 

DSA 

Detect, Sense, and Avoid 

ELOS 

Equivalent Level of Safety 

FAA 

Federal Aviation Administration 

FAR 

Federal Aviation Regulation (Title 14 Code of Federal Regulations) 

fpm 

Feet per Minute 

GA 

General Aviation 

GCS 

Ground Control Station 

GPS 

Global Positioning System 

HALE 

High Altitude, Long Endurance 

ICAO 

International Civil Aviation Organization 

IFR 

Instrument Flight Rules 

IMC 

Instrument Meteorological Conditions 

LOS 

Line-of-Sight 

MOE 

Measure of Effectiveness (Black Box. . . Top Level) 

MOP 

Measure of Performance (White Box... Lowest Level) 

NAS 

National Airspace System 

Nm 

nautical miles 

NMAC 

Near Midair Collision 

OPEC 

USAF Optical Encounter Model 

OTH 

Over-the-Horizon 

RA 

Resolution Advisory (assoc, w/ TCAS) 

ROA 

Remotely Operated Aircraft (synonymous w/ UAS) 

SUA 

Special Use Airspace 

TA 

Traffic Advisory (assoc, w/ TCAS) 

TCAS 

Traffic Alert Collision Avoidance System 

TtC 

T ime-to-Collision 

UA 

Unmanned Aircraft 

UAS 

Unmanned Aircraft System 

UAV 

Unmanned Aerial Vehicle 

USAF 

United States Air Force 

VFR 

Visual Flight Rules 

VMC 

Visual Meteorological Conditions 
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APPENDIX A: POTENTIAL APPROACHES FOR DEFINING ELOS 


This appendix provides a survey of potential methods to examine the see and avoid issue, and to 
derive measures for use in defining an ELOS see and avoid capability for unmanned aircraft 
systems. 

There are two general approaches that may be used to determine whether an unmanned aircraft is 
as safe as a manned aircraft. The first approach compares the statistics of a manned system to that 
of the unmanned system, whereas the second approach compares the performance of the manned 
system to that of the unmanned system. While both approaches can provide meaningful insight, 
only the performance approach defines ELOS using leading indicators to measure ROA 
capability. The statistics approach, on the other hand, relies upon lagging indicators that can only 
be validated after a sufficient amount of ROA operations statistics are collected. Both approaches 
will be discussed in the following sub-sections. 

A.l Statistical Approach 

A statistical approach attempts to quantify ELOS by using the manned-aviation collision statistics 
gathered over the past decade or more. The assertion is that an “equivalent level of safety” can be 
achieved if the total rate of manned and unmanned aircraft mid-air accidents, with ROA operating 
in the NAS, does not increase above the rate of just manned aircraft accidents prior to the 
introduction of ROA into the NAS. 

A. 1.1 Statistical Approach Methodologies 

The following sections discuss two methodologies within the statistical approach for establishing 
an ELOS definition. The first methodology is based on the target level of safety number 
established by ICAO for separation minima and the second methodology is based on general 
aviation mid-air collision statistics. Either methodology could be used as part of a risk ratio to 
compare manned aircraft see and avoid requirements to UAS sense and avoid requirements. 

A.l. 1.1 Method 1: Overarching Target Level of Safety Number 

Supporters of this method suggest a single number defining the overarching target level of safety 
for the air traffic management system is all that is needed to define ELOS for see and avoid. The 
target level of safety (TLS) number that would be used for this methodology is 2.5xl0' 9 , which is 
derived from ICAO Doc 9689, Manual on Airspace Planning Methodology for the Determination 
of Separation Minima [8], 

According to ICAO Doc 9689, collision risk in a given airspace is directly affected by the 
capability of ATC to detect aircraft on conflicting tracks and to correct the situation before a 
collision can occur. In order to evaluate the estimate of collision risk, it must be compared to a 
maximum tolerable collision risk for the entire system. This maximum tolerable risk is normally 
expressed in terms of a target level of safety expressed in terms of the number of fatal accidents 
per flight hour. Section 6.14 of Doc 9689 states that: for the North Atlantic airspace, the TLS for 
reduced vertical separation minimum (RVSM) levels is 5.0 x 10~ 9 and 2.0xl0' 9 for non-RVSM 
levels. Both of these values take into account the risk of collision associated with performance of 
the onboard equipment as well as that associated with ATC or pilot errors (operational errors). 
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Section 6.15 of Doc 9689 then arbitrarily sets a TLS of 2.5 xlO' 9 fatal accidents per flight hour for 
the risk of collision associated with performance of the onboard equipment. This smaller TLS 
value does not include the risk arising from either ATC or pilot errors, and is one-half of the 
original TLS number (5.0xl0' 9 ) proposed for RVSM levels. 

It should be noted, that since RVSM levels have only been implemented between FL290 and 
FL410, the TLS value of 2.5xl0~ 9 would only apply to aircraft capable of RVSM and operating 
within that altitude range. For aircraft operating above FL410 or below FL290, the overall TLS 
value of 2.0xl0" 9 would apply. If the same arbitrary value of Vi is applied to the equipment 
performance as was done for the RVSM levels, then it could be inferred that 1.0x1 O' 9 should be 
attributed to the non-RVSM aircraft’s onboard equipment. 


A. 1.1. 2 Method 2: General Aviation Mid-Air Collision Statistics 


This proposed methodology for defining ELOS uses the 
historical midair collision statistics for manned aviation. 
Since it is not envisioned that ROA will initially be used 
for transporting personnel, as commercial airliners do, 
this method specifically looks at the collision statistics 
for general aviation (GA) aircraft. Figure Al and Table 
Al, published within the 2003 Nall Report [9] and 
NTSB Aircraft Accident report [10], provide the U.S. 
general aviation accidents per 100,000 flight hours for 
the 10 year period from 1993 to 2002. Using this 
approach, an equivalent level of safety could be achieved 
if the average number of midair collisions per 100,000 
hours did not increase for the year that ROA are allowed 
to fly within civil airspace. Using the data provided in 
Table 1, one would expect an equivalent level of safety 
to be achieved if the rate of mid-air collisions, using 
manned and unmanned collision statistics, remains at or 
below 0.052 mid-airs per 100,000 flight hours. 


U.S. GENERAL AVIATION ACCIDENTS 

PM 100.000 FBght Hours 1093-2002 



Figure Al : GA Midair Collision 
Statistics, 1993 - 2002 


Table Al: General Aviation Mid-air Collision Statistics, 1993 to 2002 
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Mid Air Collisions 


Mid. 

per 10 
Flight l 

Airs 

0,000 

Hours 

Year 

All 

Fatal 

Flight Hours 

All 

Fatal 

1993 

13 

7 

22,796,000 

0.057 

0.031 

1994 

11 

7 

22,235,000 

0.049 

0.031 

1995 

14 

8 

24,906,000 

0.056 

0.032 

1996 

18 

6 

24,881,000 

0.072 

0.024 

1997 

13 

11 

25,591,000 

0.051 

0.043 

1998 

14 

11 

25,518,000 

0.055 

0.043 

1999 

15 

7 

29,246,000 

0.051 

0.024 

2000 

19 

11 

27,838,000 

0.068 

0.040 

2001 

5 

3 

25,431,000 

0.020 

0.012 

2002 

9 

5 

25,545,000 

0.035 

0.020 

AVG 

13.10 

7.60 

25,398,700 

0.052 

0.030 
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A.1.2 Benefits of using the Statistical Approach 

This approach attempts to assist regulators in establishing the minimum operational performance 
standards and specifications for a collision avoidance system in a simple measure. The only 
measure to be made is whether or not the total number of mid-air accidents per year increase after 
ROA have been introduced into the NAS. The benefit, therefore, would be a lesser workload on 
regulators since there wouldn’t be a need to generate numerous standards and specifications other 
than the single statistical number used to define ELOS. 

Another benefit of this approach is that it could be used to verify or validate an ELOS definition 
based on leading indicators such as performance-based items discussed in section A. 3. In this 
capacity, the statistical number could be used as a lagging indicator to either confirm the 
performance-based ELOS indicators are satisfactory or if there is a need for performance tradeoffs 
in a previously approved design. 

A.1.3 Drawbacks with the Statistical Approach 

One concern with the statistical approach is that the final value, which gets proposed as being an 
equivalent level of safety to manned aviation, will be difficult to validate prior to the insertion of 
ROA into the NAS. Reliability and statistical numbers, by definition, require data to be gathered 
over a long duration of time. Prior to collection of data over a statistically meaningful period of 
time, one can only predict results through analysis, simulation, or demonstration through 
similarity. 

Another concern with this approach is the complexity of mid-air collision statistics. Results can 
be skewed by a number of factors, e.g. ROA encounter scenarios may be different from those of 
manned aircraft, evolution of the NAS, and variability of mid-air collision statistics from region to 
region. As an example of the different encounter probabilities, the AOPA Air Safety Foundation 
2001 Nall Report [7] says that 80% of all mid-air collisions are within 10 miles of an airport while 
in the traffic pattern, 77% occurred at or below 3,000 feet AGL, and 49% occurred at or below 
500 feet AGL. With automated takeoff and landing and no requirement for proficiency training in 
the traffic pattern, ROA operating in this airspace far less frequently than manned aircraft could 
mask a higher ROA mid-air collision rate in other flight regimes if only the overall rates are 
examined. 


A.2 Performance-Based Approach 

A performance-based approach for defining ELOS could have a higher chance of success for 
being used by the FAA to establish collision avoidance requirements for ROA. Collision 
avoidance requirements derived from a performance-based approach can also provide ROA 
platform developers and sensor designers with a set of measurable performance parameters that 
could ultimately be used to verify that their proposed system has an equivalent level of safety to 
that of a manned aircraft. Establishing these measurable parameters will provide the FAA with a 
“ leading indication ” the ROA system is suitable to fly within the NAS; thereby giving them 
greater confidence the system will be capable of maintaining safe separation. 
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A.2.1 Performance-Based Approach Methodologies 

A performance-based approach attempts to quantify ELOS by using either 1) the physical and 
mental limitations of the human pilot or 2) the existing see and avoid rules and guidelines already 
in place for manned aircraft. The assertion is that an equivalent level of safety can be achieved by 
developing a see and avoid system that meets or exceeds the capabilities of a human pilot or 
complies with the existing FAA rules and guidelines for avoiding midair collisions in manned 
aircraft. Both of these methods are further described below. 

A.2.1.1 Method 1: Human Pilot Limitations, Regarding See and Avoid 

The first performance-based method attempts to bound the problem by focusing on the physical 
and mental limitations common to all human pilots as they perform the task of collision 
avoidance. Over the past several decades, many individuals and groups have attempted to 
quantify the temporal and visual capabilities of humans as they relate to see and avoid. In addition 
to modeling and simulation work, many of these results have also been substantiated over the 
years through various tests and experiments that have focused on several parameters. A 
discussion of several of these human pilot see and avoid limitations follows. 

In order for pilots to avoid midair collisions, they must perform a number of steps. These steps 
involve scanning the surrounding airspace for other aircraft, determining if the detected aircraft is 
on a collision trajectory, deciding what evasive maneuver must be performed, if any, and initiating 
the appropriate maneuver allowing sufficient time for the pilot’s own ship to respond and maintain 
separation. This process of detecting and avoiding other aircraft is comprised of six essential 
functions, which must be performed in the appropriate order to lessen the chance of a mid-air 
collision occurring. These six DSA functions are depicted below and listed in Figure A2. 
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PSA Functions 


Detect 

-Detect Object 

See 

-Track Object 

-Identify Collision Potential 


Avoid 


-Decide on Evasive Maneuver 
-Initiate Avoidance Maneuver 
-Complete Avoidance Maneuver 



f Initiate 
avoidance 
^maneuver(s) 




F5 


F4 



F3 


Identify 

collision 

potentials) 



Figure A2: The Functions Comprising Detect, See, and Avoid 


It is necessary to always think of collision avoidance, as a process comprised of these six 
functions, regardless of whether it is for a manned or unmanned aircraft. Each of these functions 
performs a critical role toward providing an overall collision avoidance capability essential to 
maintaining safe and reliable operations as a function of see and avoid. 

Pilot Reaction Time 

One of the most widely accepted studies attempting to quantify how long it takes a human pilot to 
perform these six functions was performed by Tyndall Air Force Base in 2001 and is documented 
within their Air Force - Mid-Air Collision Avoidance (AF-MACA) Document [11]. This study 
was performed using military pilots flying military jet aircraft with closing speeds up to 1,100 
mph. According to this study, a duration of 12.5 seconds is typically required for a pilot to detect 
an intruding aircraft, track the intruder aircraft’s flight path, determine the collision potential with 
the other aircraft, decide upon an 
evasive maneuver, and then 
actually initiate the evasive 
maneuver. Figure A3 depicts all 
6 of the DSA functions and the 
time allocated to each within the 
study. Note that the 12.5 seconds 
is only for the first five functions. 

These are the functions that are 
dependent upon the human pilot, 
whereas, the time to complete the 
maneuver is dependent upon the 
performance capability of the 
aircraft, the type of maneuver 



Nv . 



Detect 

0.1 sec 

Track 

1.0 sec 

Identify 

5.0 sec 

Decide 

4.0 sec 

Initiate 

+ 2.4 sec 


12.5 sec 


Figure A3: Human Pilot Collision Avoidance Timeline 


performed (i.e. climb or dive, heading change, or combination of these two), and the 
aggressiveness of the maneuver. 

A second source for these times was found after discussions with the Air Force personnel involved 
in the Tyndall AFB test. From this conversation, a book called Human Factors in Aviation [12] 
was mentioned. Upon reviewing this book, a statement was found which said: “ Estimates of the 
time required to recognize an approaching aircraft and take evasive action range from 5.0 to 12.5 
seconds...” However, no supporting analysis was provided other than it was based on military 
tests previously performed. 

A third study performed by the University of California in 1972, alluded to the time necessary to 
perform collision avoidance as 10 seconds. This study, Visual Aspects of Air Collision 
Avoidance [13], states: “Ten seconds has sometimes been stated as the minimum required time for 
visual acquisition if successful evasive action is to result .” 

Regardless of whether 10 seconds or 12.5 seconds is used to begin quantifying the time it takes a 
human pilot to perform collision avoidance, the total time needed to avoid must also include the 
aircraft response time. For an ROA this will not only depend upon the maneuverability of the 
aircraft, but also upon the latency and availability of the command and control link. 

Visual Scan Persistence 

Obviously, see and avoid will only work if the pilot is looking outside the cockpit. According to a 
study performed in 1976, U.S. pilots on VFR flights spend about 50 percent of their time in 
outside traffic scan [14]. Even more disturbing, are the results from a similar study done 
regarding commercial airline pilots. This study suggests airline pilots spend about 20 percent of 
their time in outside scan [15]. Of course, the time spent scanning for traffic is likely to vary with 
traffic density and the pilot’s assessment of the collision risk [16] 

Detection Range 

As discussed in Section 2.3, several factors can make visually detecting other aircraft more or less 
challenging. Some of these factors may include: the relative motion of the approaching aircraft; 
the aspect of the aircraft; or the approaching aircraft’s contrast against its background. Factors 
such as aircraft color, lighting color, and strobe lighting can all aid a pilot in his/her ability to 
make an initial detection. Several studies have been done to assess the detection range at which a 
pilot is first able to detect aircraft on a collision approach. Four of these studies are discussed 
below. 

One of the first analytical studies performed on this topic was conducted at the University of 
California in 1972. This analysis used a Cessna 180 and a Douglas DC-8 for determining 
probability of detection based upon time until impact for a variety of collision geometries. The 
nominal human detection range resulting from this analysis was 2 nautical miles (nm). In 
addition, this study showed overwhelming proof that the visual detection performance of a pilot 
can be significantly improved if he knows when and where to look. [13] 
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In the late 1970s through mid 1980s, MIT Lincoln Laboratory developed an independent see and 
avoid model in support of the TCAS initiative. This model was based upon several series of flight 
experiments, which produced a database of pilot visual acquisition performance for a variety of 
near collision encounters. These experiments, which involved a Cessna 421 and Beech Bonanza, 
found that the median acquisition range for “un-alerted pilots” was 0.99 nm and 1.14 nm when 
pilots were alerted to the approximate bearing of the target [17], This model was able to also 
show that the probability of acquiring an intruder was largely dependent upon the workload 
devoted to the search and the uncertainty in the approach bearing of the target aircraft. 

A third study exploring human detection range was performed by the Air Force Research 
Laboratory (AFRL) in 2002, using the Optical Encounter (OPEC) model. This model emulates 
the human ability to detect aircraft under a variety of environmental conditions, and has been 
validated by 937 trials under all different lighting conditions and aspect scenarios. The results of 
this study indicated that the human detection range for an alerted pilot was an average of 1.6 nm, 
+/- 0.2 nm [18] against military aircraft, assuming a 90% detection probability and a +/- 15 degree 
search cone. 

The fourth and final activity to be discussed here was performed by the NASA Environmental 
Research Aircraft and Sensor Technology (ERAST) Project. Over a two year period, between 
2001 and 2002, The NASA ERAST project sponsored several collision avoidance flight 
demonstrations intended to assess the feasibility of multiple technologies for see and avoid. The 
NASA ERAST Non-cooperative PSA Flight Test Report states, “the typical detection range for 
two human pilots was between 1.0 and 1.5 nautical miles' 1 ’ 1 [19]. This detection range was the 
result of nearly 50 collision scenarios, in which both pilots onboard the aircraft knew the exact 
time and direction from which the intruder aircraft would be approaching. 

It is important to note, that all four of these completely independent tests achieved surprisingly 
similar values for human detection range for alerted and un-alerted pilots. Also noteworthy is the 
fact that two of these studies involved analytical modeling tools and two involved actual flight 
tests. 

Probability of Detection 

What is the likelihood that a pilot will detect another aircraft approaching on a collision trajectory? 
The probability of detection (Pd) for a given collision scenario will largely depend upon the 
workload and attentiveness of the pilot, the approaching aircraft’s contrast with the background, 
and whether or not the pilot was previously alerted regarding when and where to look. With 
respect to the Lincoln Labs flight test (discussed earlier), the pilots, who were un-alerted, 
detected the approaching aircraft in thirty-six out of sixty-four encounters, or 56% [17], With 
respect to NASA ERAST flight tests, the pilots were alerted when and where to look. As a result, 
the ERAST pilots were able to visually acquire the approaching aircraft prior to the 1 nm “call- 
off’ distance in 46 of the 50 encounters, or 92%. This value is similar to a study performed by 
AFRL, where they indicate that a 90% Pd is reasonable and comparable to human performance 
when the pilot is alerted about the approaching intruder aircraft [18]. 

Update Rate 
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The individual eye movements associated with visual search takes a small but significant amount 
of time. According to Applied Optics [20], the most updates the human eye can make are 3 
fixations per second. This can, however, substantially increase if the pilot is scanning a complex 
scene. 

Visual Acuity 

There are times when the approaching aircraft will be too small to be seen because it is below the 
eye’s threshold of acuity. According to the book Modem Optical Engineering by Warren Smith 
[23], “ The normal, healthy, unaided human eye has a resolution of about 

0.3 milliradians'’'’', or about 0.02 degrees. This high level of acuity, however, only applies to the 
foveal 1° cone, where vision is the best. Outside of a 10° cone, concentric to the foveal 1° cone, 
human vision acuity decreases to one-tenth of it’s foveal value [21]. In terms of an oncoming 
aircraft, if a pilot is able to see an aircraft at 4,000 feet within his/her foveal field, with peripheral 
vision the aircraft wouldn’t be detected until it was only 400 ft away. 

While aircraft on converging tracks will appear 
larger as they get closer, it can be said that the 
apparent size of an on-coming aircraft doubles 
with each halving of that aircraft's range. 

Imagine the case in which a general aviation 
aircraft and a military jet are approaching each 
other head-on at speeds of 150 knots and 450 
knots respectively, a closing speed of 600 knots. 

At about 20 seconds before impact, the two 
aircraft might be about 3200 ft apart and each 
will present a target to the other of only around 
1/1 6 th of a degree. Ten seconds from impact, the 
distance will be halved and the target size will 
have increased to all of l/8 th of a degree; at 5 
seconds, the size will have again doubled but 
will still be only about 1/4 of a degree. In other 
words, the oncoming aircraft remains extremely 
small until very late, and then it suddenly 
expands into something that fills the windscreen. 

Figure A4 illustrates this concept [22]. 

Field of Regard 

Field of regard (FOR) is the volume of sky that a human can see, in both the horizontal and 
vertical positions. According to the Civil Aero-Medical Institute (CAMI), the normal vertical 
vision field is 135°, 60° up and 75° down. The horizontal vision field, according to CAMI, is 
+/- 100° left and right. Both of these volumes are for a stationary head, using both eyes. 
Obviously, by moving one’s head left/right or up/down, the total FOR can greatly be increased. 
However, the aircraft cockpit and structure usually restrict the benefits of this substantially (unless 
of course it is a fighter jet with a large transparent canopy). 



Figure A4: Time to impact and angular size 
of oncoming aircraft Timeline [ASTB] 
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Another source for horizontal FOR is the Bureau of Air Safety Investigation (BASI), which states 
that “The average person has a field of vision of around 190 degrees ...” which equates to +/- 95° 
azimuth. Regardless of the FOR being scanned, some areas of the visual field receive close 
attention while other areas are neglected. For example, areas of the sky near the edges of the 
windscreen are generally scanned less than the sky in the center of the windscreen [20], 

Field of View 

Although a human’s field of regard covers a rather large volume of sky, he/she is likely only to 
detect aircraft that they are directly looking at. According to BASI [16], a human can only focus 
on a 10° to 12° area at a time. On both sides of this area, all human eyes have blind spots which 
greatly prohibit viewing anything within a 6° region between 12° and 18° to the left and right of 
the foveal 1° cone. 
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Workload 


The limiting mental processing capacity of the human pilot can present problems when there is a 
need to attend to two or more sources of information at the same time. This obviously pertains to 
performing in-cockpit related activities such as monitoring the instruments and out-of-cockpit 
activities such as scanning. However, even if pilots could spend 100% of their time looking for 
traffic, they can get overwhelmed if multiple aircraft are detected and each need to be tracked for 
determining collision potential. A number of researchers have shown that peripheral stimuli are 
more difficult to detect when attention is focused on a central object or task [23], For example, 
while a pilot is concentrating on one aircraft, a second aircraft could enter their peripheral vision 
and never be detected until too late. 

Other limiting factors 

Other factors known to affect vision include alcohol, tobacco, stress, sleep deprivation, low 
barometric pressure, windscreen haze, improper illumination of the cockpit and/or instruments, 
and dirty instruments [21], 

Summary 

Table A2 summarizes the results discussed in this section. 


Table A2; “Equivalent Level of Safety” Based Upon a Human Pilot’s See & Avoid Limitations 


Parameter 

Value 

Reference/Source 

Pilot Reaction Time 

10 sec 
12.5 sec 

“Visual Aspects of Air Collision Avoidance” [13] 
“Mid-Air Collision Avoidance” [11] 

Percent of time spent 
scanning 

GA: 50% 
Airline: 20% 

“Visual Attenuation of Private Pilots, the Proportion of time 
Devoted to Outside the Cockpit” [14] 

Detection Range for 
Alerted Pilots 

1.14 nm 
1.6 nm 
1.5 nm 

Lincoln Labs TCAS Flight Tests [17] 

AFRL OPEC Model “See & Avoid for UAV’s” [18] 
NASA ERAST See & Avoid Flight Test [19] 

Probability of 
Detection (Pd) 

Un-alerted: 56% 
Alerted: 92% 

Lincoln Labs TCAS Flight Tests [17] 

NASA ERAST See & Avoid Flight Tests [19] 

Update Rate 

3 fixations/sec 

“Ocular Behavior in Visual Search” [20] 

Visual Acuity 

0.3 mrad 

“Modern Optical Engineering” [23] 

Field of Regard 
(FOR) 

Vertical: +60°, -75° 
Horizontal: +/-100° 

“Pilot Vision Brochure” [21] 

Field of View 
(FOV) 

10° - 12° cone 

“Limitations of the See & Avoid Principle” [16] 
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A.2.1.2 Method 2: Existing Aviation Regulations 

The second performance-based method for defining equivalent level of safety relies on the FAA 
and ICAO guidelines that have already been established for the safe operation of manned aircraft. 
Many of the requirements imposed on manned aviation could potentially be used to establish the 
requirements for a ROA collision avoidance system initial capability. This approach adopts the 
existing rules and guidelines that pertain to see and avoid. If a sensor can provide a similar 
capability for detecting and avoiding collisions, then it could be argued that an equivalent level of 
safety for see and avoid has been reached. The following paragraphs will identify what some of 
these existing rules and guidelines state with respect to pertaining to see and avoid. 

Pilot Reaction Time 

With respect to time-to-collision, the FAA guidelines support the analysis and studies mentioned 
above in the human-limitation discussions. According to FAA-P-8740-51, FAA Accident 
Prevention Program [25], “It takes a minimum of 10 seconds for a pilot to spot traffic, identify it, 
realize it is a collision threat, react, and have his aircraft respond.'” 

Similarly, Advisory Circular (AC) AC 90-48C, entitled Pilot’s Role in Collision Avoidance shows 
a graphic supporting the Tyndall AFB-MACA number of 12.5 seconds. This graphic allocates 0. 1 
sec to seeing the object, 1.0 sec to recognizing the aircraft, 5.0 sec for becoming aware of the 
collision course, 4.0 sec for the decision to turn left or right, 0.4 sec for muscular reaction, and 2.0 
sec for aircraft lag time [26]. 

Separation Minima 

All aircraft must be capable of maintaining adequate separation with other traffic. FAA Order 
8020.11 identifies a Near Mid Air Collision (NMAC) as an “ incident ” [27], and FAA Order 
8700.1 sets out guidelines for classifying the hazard level of NMACs [28]. According to 8700.1, 
the hazard level based on miss distance is as follows: 

Critical Hazard: <100 feet 

Potential Hazard: 100 - 500 feet 

No Hazard: > 500 feet 

Therefore, to ensure a NMAC or incident does not occur, all aircraft must maintain a 500-foot 
radius safety bubble around their vehicle. 

Visibility 

The minimum visibility requirements for operating an aircraft under visual flight rules (VFR) 
conditions are established in 14 CFR 91.155. According to this source, the visibility requirement 
to operate below 10,000 ft is 3 statute miles and 5 statute miles when above 10,000 ft [29]. It is 
important to note that these visibility requirements are not intended to identify a range for 
detecting approaching traffic, but rather are simply a minimum visibility distance. Although 
visibility and detection range are not equivalent, these visibility requirements could be used to 
establish the maximum required by the FAA to operate within the NAS under VFR conditions. 
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Azimuth Field of Regard (FOR) 


The FAA does not require a specific azimuth of visibility 
from the cockpit, but in general, a pilot would need to see 
beyond 90 degrees right and left, especially to check the 
airspace prior to a turn. Also, the FAR right-of-way rules 
[2] require the converging aircraft to deviate to the right to 
avoid head-on conflicts, or deviate to the right to pass 
behind a conflicting aircraft approaching from the right. It 
is also the responsibility of an overtaking aircraft to avoid 
a slower air vehicle. 

The ICAO Right of Way Rules (Annex 2 ) [6] state that 
“an aircraft intercepting from an aspect angle +/- 70 
degrees from the tail, must give way to the other aircraft' 
A graphic of this is shown in Figure AS. The inverse of 
+/- 70° from the tail is obviously +/- 110° from the nose of 
the aircraft. Therefore, a Field of Regard azimuth of +/- 
110 degrees should be sufficient to provide equivalent 
detection relative to a manned aircraft. 



Figure A5: ICAO Annex 2, 
Azimuth FOR Responsibility 


The FAA Air Safety Foundation pilot information pamphlet, How to Avoid a Midair Collision 
also provides guidance on azimuth FOR. This pamphlet suggests that “ looking +/- 60 degrees off 
the nose would catch the vast majority of conflicting aircraft " [25], 

A third reference that discusses azimuth FOR is AC 25.773-1, Pilot Compartment View Design 
Considerations . This Advisory Circular states: “ vision through the transparent areas should 
provide the following pilot compartment view ...120° left to ... and 120 ° right” [30], It is 
important to note, however, that this document is only intended to provide guidance on cockpit 
design, and not to provide guidance for avoiding collisions. In fact many aircraft do not adhere to 
this AC, while others exceed the suggested amount of transparent material for cockpits. 

Elevation Field of Regard IFOR) 

Although the FAR does not require a specific 
elevation of visibility from the cockpit, it does 
require that climbing/diving aircraft give way to 
air vehicles above/below [2], The practical 
requirement for vertical visibility is dependent on 
aircraft climb/dive performance and operation 
where an aircraft that operates with high rates of 
vertical maneuver would require a higher 
elevation scan capability than an aircraft with a 
slower rate of vertical maneuver. The FAA and 
Air Safety Foundation pilot information 
pamphlet, How to Avoid a Midair Collision, states that “ looking +/- 10 degrees in elevation would 
catch, virtually all conflicting aircraft' [25]. Figure A 6 depicts this elevation field of regard. 



Figure A6: FAA Recommendation for 
Elevation FOR 
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Lastly, AC 25.773-1, Pilot Compartment View Design Considerations, also discusses the elevation 
FOR. This Advisory Circular states: “Vision through the transparent areas should provide the 
following pilot compartment view ...forward and up 35 degrees... to forward and down 27 
degrees” [30], As noted previously when this reference was cited in the azimuth FOR section, 
these values are intended for cockpit design, not collision avoidance constraints. 

Field of View (FOV) 

The FAA Accident Prevention Program document, How to Avoid a Midair Collision [25], 
provides insight into the ability of a human to focus on an area of airspace while searching for 
approaching aircraft. This document states: “The human eye is limited to a relatively narrow area, 
approx. 10-15 degree, in which it can actually focus and classify an object ”. 

Update Rate 

Based upon the 10°- 15° field of view limitation for the human eye, which was discussed in the 
previous paragraph, the FAA established guidelines for scanning to detect approaching aircraft. 
FAA-P-8740-51 states: “By fixating every 10 - 15 degrees, you should be able to detect any 
contrasting moving object in each block. This gives you 8 to 12 blocks in your scan area, each 
requiring a minimum of one to two seconds for accommodation and detection [25] Since this 
same document also suggests a scan area of +/-60 degrees azimuth by +/-10 degrees elevation, the 
suggested update rate can therefore be determined. If the suggested search volume is comprised of 
12 blocks, and if one to two seconds are spent in each block, then the total amount of time needed 
to scan this volume once would be between 12 seconds and 24 seconds depending upon whether 
one or two seconds were spent searching in each block. 

Summary 

Table A3 summarizes the results discussed in this section. 


Table A3; “Equivalent Level of Safety” Based Upon FAA & ICAO References 


Parameter 

Value 

Reference/Source 

Pilot Reaction 

10 sec 

FAA-P-8740-51 “How to Avoid a Midair Collision.” [25] 

Time 

12.5 sec 

AC 90-48C “Pilots’ Role in Collision Avoidance” [26] 

Missed Distance 

> 500 ft is considered 
“No Hazard” 

FAA Order 8700.1 “Investigate a Near Midair Collision” [28] 

Visibility 

3 mi, below 10K ft 
5 mi, above 1 OK ft 

FAR 91 .155 “Minimum visibility for VFR flight” [29] 

Azimuth FOR 

+/- 60° 

FAA-P-8740-51 “How to Avoid a Midair Collision.” [25] 


+/- 110° 

ICAO Annex 2 “Right of Way Rules” [6] 


+/- 120° 

FAA AC 25.773-1 “Pilot Compartment Design” [30] 

Elevation FOR 

+/- 10° 

FAA-P-8740-51 “How to Avoid a Midair Collision.” [25] 


+37° to -25° 

FAA AC 25.773-1 “Pilot Compartment Design” [30] 

Scan Rate 

12 sec to 24 sec 

FAA-P-8740-51 “How to Avoid a Midair Collision.” [25] 
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A.2.2 Benefits of using the Performance-Based Approach 

This approach attempts to quantify equivalent level of safety at a level that contains technically 
meaningful and/or more verifiable data. Establishing performance-based criteria would entail the 
development of collision avoidance requirements which could then potentially be used by the 
FAA in establishing minimum aircraft system performance standards (MASPS), minimum 
operational performance standards (MOPS), or technical standard orders (TSO) for see and avoid 
systems. 

The major benefit to using this approach is that a performance-based ELOS definition will provide 
a leading indication to operators and regulators that the proposed system will in fact meet or 
exceed a human’s capabilities. For example if a certain field of regard and detection range were 
required to meet an equivalent level of safety, then an adequate number of simulations and flight 
tests could be performed to verify the sensor could in fact meet these requirements. If these 
parameters were successfully met within the simulations and flight tests, the regulators would 
have a high degree of confidence in the see and avoid system before ever permitting it to operate 
within the NAS. 

A.2.3 Drawbacks with the Performance-Based Approach 

One concern with a performance-based approach is that multiple parameters may need to be 
specified, as opposed to a single number proposed under the statistical approach. Each of these 
parameters could not only require a substantial amount of analysis to derive the necessary 
requirements, but could potentially conflict with each other if appropriate trade offs are not made. 

The biggest concern with this approach is that the requirements will not take into account the 
unique maneuverability characteristics of each ROA. If the unique characteristics of ROA are not 
considered when developing the requirements, all ROA could be required to carry the identical see 
and avoid sensor, which would have the capability to avoid collisions based on the worst case 
scenario for the fastest moving, least maneuverable aircraft. If all ROA were required to carry an 
identical see and avoid sensor to operate within the NAS, there would be a high probability that 
some ROA wouldn’t be able to carry such a system due to size, weight and power constraints. 
Supporters of this approach, therefore, obviously desire to have the ROA’s unique characteristics 
be taken into account when developing requirements so as not to over-specify the see and avoid 
sensor necessary for that vehicle to operate within the NAS. What is needed by one ROA to 
ensure collision avoidance may be over-specified for a slower moving, more maneuverable ROA. 


A.3 Summary of the Potential ELOS Approaches 

As one can see from the information provided within the previous sections, the term “equivalent 
level of safety” can be defined using several different approaches and methodologies, each based 
on a different premise. The statistical approach advocates using one number to define ELOS. 
This number is either the overarching safety number (5.0x10-9) or the average number of mid-air 
collisions for manned general aviation aircraft (average of 13.1 mid air collisions per year over 
past ten years) provided in Table Al. The performance-based approach, on the other hand, 
advocates establishing the ELOS definition that is based upon either human pilot’s physical and 
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mental limitations or upon the existing manned aviation regulations found within the FAR, ICAO 
Annex2, and various FAA Orders and Advisory Circulars. Several performance parameters and 
their potential values for these two methodologies have been provided within Tables A2 and A3 
above. 
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